Toward Faster Packing and Unpacking of MPI Datatypes

The Message Passing Interface (MPI) standard provides a powerful mechanism for describing non-contiguous memory locations: derived datatypes. In addition, MPI derived datatypes have a key role in the MPI-2 I/O operations. In principle, MPI derived datatypes allow a user to more efficiently communicate noncontiguous data (for example, strided data) because the MPI implementation can move the data without any intermediate copies to or from a contiguous buffer. In practice, however, few MPI implementations provide support for datatypes that performs better than what the user can achieve by manually packing and unpacking the data into contiguous buffers before calling MPI routines with contiguous memory regions. We develop a taxonomy of MPI datatypes according to their memory reference patterns and show how to efficiently implement these patterns. The effectiveness of this approach is illustrated on a variety of platforms.